Applications of Empirical Processes in Learning Theory: Algorithmic Stability and Performance Guarantees
نویسندگان
چکیده
First, we demonstrate how the Contraction Lemma for Rademacher averages can be used to obtain tight performance guarantees for learning methods [3]. In particular, we derive risk bounds for a greedy mixture density estimation procedure. We prove that, unlike what is suggested in the literature, the number of terms in the mixture is not a bias-variance trade-off for the performance. Our upper bound asserts that the estimation error decreases as the number of terms in the mixture increases.
منابع مشابه
Applications of Empirical Processes in Learning Theory: Algorithmic Stability and Generalization Bounds
This thesis studies two key properties of learning algorithms: their generalization ability and their stability with respect to perturbations. To analyze these properties, we focus on concentration inequalities and tools from empirical process theory. We obtain theoretical results and demonstrate their applications to machine learning. First, we show how various notions of stability upperand lo...
متن کاملProbabilistic Sufficiency and Algorithmic Sufficiency from the point of view of Information Theory
Given the importance of Markov chains in information theory, the definition of conditional probability for these random processes can also be defined in terms of mutual information. In this paper, the relationship between the concept of sufficiency and Markov chains from the perspective of information theory and the relationship between probabilistic sufficiency and algorithmic sufficien...
متن کاملGeneralization Bounds of Regularization Algorithms Derived Simultaneously through Hypothesis Space Complexity, Algorithmic Stability and Data Quality
A main issue in machine learning research is to analyze the generalization performance of a learning machine. Most classical results on the generalization performance of regularization algorithms are derived merely with the complexity of hypothesis space or the stability property of a learning algorithm. However, in practical applications, the performance of a learning algorithm is not actually...
متن کاملStability of Controllers for Gaussian Process Dynamics
Learning control has become an appealing alternative to the derivation of control laws based on classic control theory. However, a major shortcoming of learning control is the lack of performance guarantees which prevents its application in many real-world scenarios. As a step towards widespread deployment of learning control, we provide stability analysis tools for controllers acting on dynami...
متن کاملExploring Language Learners’ Cognitive Processes in On-line ESP Courses via Think-aloud Protocol Analysis
The present study aims to investigate language learners’ cognitive processes in on-line ESP courses. Three modes of inquiry are used: think-aloud protocol analysis, screen capture analysis, and correlation analysis. The theoretical foundations for the evaluation of the cognitive aspect of Ferdowsi Univeristy of Mashhad E-learning System are drawn from cognitive load theory, cognitive apprentice...
متن کامل